Digital watermarking refers to the embedding of a hidden message in an image or image sequence for such purposes as establishing ownership, tracking the origin of the data, preventing unauthorized copying, or conveying additional information (meta-data) about the content. Watermarking has potential uses in a wide range of products, including digital still and video cameras, printers and other hardcopy output devices, and content delivery services (e.g., Internet-based photofinishing). Recently, there has been significant interest in the electronic distribution and display of theatrical movies, which is termed digital cinema. Studios and distributors have a strong need to protect the movie content from unauthorized use, and watermarking can assist by establishing ownership and tracing the source of stolen content (through the use of hidden date/time/location stamps inserted at the time of the movie distribution and/or presentation). A related application is the distribution of digital movie content into home environments, where a person could download a movie via a broadband connection (such as a cable modem). Again, watermarking can provide a means for tracing the movie content through authorized and unauthorized distribution channels. The present invention relates specifically to the extraction of watermarks in image sequences, and thus it has usefulness in applications such as digital cinema and home delivery of movies.
In these applications, the data that represents image sequence may undergo numerous degradations prior to any attempt to extract the watermark. These degradations can occur because the data was compressed for more efficient storage and transmission; it was converted to an analog signal at some point (such as when recording on a VCR); or it was captured by a camcorder in a movie theater (e.g., pirated material). Such degradations make it significantly more difficult to successfully extract the embedded watermark, regardless of the type of watermarking system that is used.
Numerous watermarking methods have been described in the prior art, including both patents and the technical literature. Many of these methods are described in review papers such as: Hartung and Kutter, Multimedia Watermarking Techniques, Proc. IEEE, 87(7), pp. 1079–1107 (1999), and Wolfgang et al., Perceptual Watermarks for Digital Images and Video, Proc. IEEE, 87(7), pp. 1108–1126 (1999). As noted in these review papers, most research on watermarking techniques has focused on single-frame images, and there are significantly fewer methods that are specific to image sequences (i.e., video watermarking). Of course, a watermarking method that has been designed for single-frame images could be applied to an image sequence by merely repeating the same process for each frame. However, this approach does not take advantage of the multiple frames that are present in an image sequence, which can be used to improve the overall performance of a watermarking system in such applications.
There are several prior art patents that include video-specific watermarking methods: U.S. Pat. No. 5,809,139 issued Sep. 15, 1998 to Girod et al. entitled Watermarking Method and Apparatus for Compressed Digital Video, B. Girod et al., Sept. 15, 1998; U.S. Pat. No. 5,901,178 issued May 4, 1999 to Lee et al. entitled Post-Compression Hidden Data Transport for Video; U.S. Pat. No. 5,991,426 issued Nov. 23, 1999 to Cox et al. entitled Field-Based Watermark Insertion and Detection; and U.S. Pat. No. 6,026,193 issued Feb. 15, 2000 to Rhoads entitled Video Steganography.
In the patents by Girod et al. and Lee et al., the methods are designed for directly embedding a watermark in compressed frequency-domain video streams (such as MPEG-encoded sequences). The patent by Cox et al. describes a method for alternately embedding positive and negative watermarks in consecutive fields of an interlaced video signal; this method is not suitable for progressively scanned image sequences such as those used in digital cinema applications. The patent by Rhoads discloses the basic concept of using multiple watermarked frames from an image sequence to extract the watermark with a higher degree of confidence than would be obtained with only a single frame. However, in the patent by Rhoads, the same watermark pattern must be used for each frame in order to combine the frames for improved performance. Particularly in watermarking system for image sequences, it may be desirable to change the watermark pattern from frame to frame for the purpose of decreasing visibility, increasing security, and/or distributing the embedded message data across multiple frames. Moreover, none of the methods described in the aforementioned patents make direct use of the repetitive nature of image content in the sequence, i.e., the high degree of correlation that exists between consecutive frames of a sequence, as a means of improving watermark extraction performance.
In a paper by Kalker and Haitsma entitled “Efficient detection of a spatial spread-spectrum watermark in MPEG video streams” (IEEE Int. Conf. on Image Processing, Vancouver, British Columbia, Sept. 2000), a method is disclosed for the computationally efficient detection of watermarks that have been embedded in an MPEG compressed video sequence. Relevant to the present invention, the authors make the observation that a residual frame in an MPEG sequence may have more favorable characteristics for watermark detection because of reduced image energy. Their method is specific to MPEG compressed frames, and the reduction in image energy for residual frames is limited by the constraints of the MPEG encoding process. Moreover, only certain frames in an MPEG sequence are residual frames, and thus the benefits of reduced image energy are only achieved in a fraction of the total number of frames. Furthermore, in heavily compressed MPEG sequences, much of the information (including both image content as well as the watermark signal) in a residual frame is discarded during the encoding process, thus lowering the effectiveness of the watermark extraction process on the residual frames.
There is a need therefore for an improved watermark extraction technique for image sequence data that: (1) increases the successful extraction rate of embedded watermarks; (2) works with any watermark embedding method; (3) does not require the same watermark pattern in each frame; and (4) can be applied to any type of image sequence data and is not constrained to MPEG compressed sequences.